feat: Add `.pyroscope.yaml` with source code mapping support #4606

simonswine · 2025-11-11T17:06:42Z

This PR adds support for .pyroscope.yaml configuration files for custom source code mappings and adds logic to use those mappings to find Go and Java code

Added .pyroscope.yaml configuration support for custom source code location mappings
Supports both local paths and GitHub repository mappings for Go and Java
Implements prefix-based matching with longest-prefix-wins strategy
Enhanced Go standard library resolution with version detection and configurable source
Added function_name parameter to file lookups, so java profiles (that don't contain file paths), can be looked up.
Improved error tracing with OpenTracing tags in GitHub client

Configuration Example (also in the PR):

This will map a special path prefix called $GOROOT to a particular go version. This signals to the backend the go version used to compile this should be kept update to date with the version of Pyroscope used to build.

  source_code:
    mappings:
      - path:
          - prefix: $GOROOT/src
        language: go
        source:
          github:
            owner: golang
            repo: go
            ref: go1.24.8
            path: src

jake-kramer · 2025-11-13T15:20:53Z

api/connect-openapi/gen/vcs/v1/vcs.openapi.yaml

          type: string
          title: rootPath
          description: the root path where the project lives inside the repository
+        functionName:


Looks like currently logic is: use localPath if go, use functionName if java. May lead to confusion down the road if we start seeing java profiles with the file paths properly set.

Yes I guess the profiling data you send needs to match with the .pyroscope.yaml you set. I do think this makes sense and is both in customer control.

examples/language-sdk-instrumentation/java/rideshare/.pyroscope.yaml

- Support golang.org/toolchain paths - Use mappings if there is no version identified

bryanhuhta

Some thoughts for the future, but I like where this is at right now.

pkg/frontend/vcs/config/config.go

pkg/frontend/vcs/source/find.go

jake-kramer

LGTM!

examples/language-sdk-instrumentation/java/rideshare/.pyroscope.yaml

jake-kramer · 2025-11-21T15:21:08Z

pkg/frontend/vcs/config/config.go

+}
+
+// Validate checks if a source configuration is valid
+func (m *Source) Validate() error {


Would be helpful error message to surface if the supplied source is not supported (i.e. not github or local)

Not fully sure, if I am understanding this comment: There are two parts:

Not surfacing the error: I agree nothing surfaces this right now to a user, I think we will need first CLI support like profilescli pyroscope-yaml validate, but it should also be possible to surface them in Drilldown, when function details is used.

"if the supplied source is not supported (i.e. not github or local)": This will trigger the error in Line 136. Not too sure what else we could do.

By the "supplied source is not supported" I meant specifically this scenario:

mappings: - function_name: - prefix: org/example/rideshare language: java source: local: path: src - function_name: - prefix: org/example/foo language: java source: gitlab: path: src

will return the error "no source type supplied, you need to supply exactly one source type" which does not let the user know that gitlab is an unsupported source

pkg/frontend/vcs/source/find.go

jake-kramer · 2025-11-21T15:33:30Z

pkg/frontend/vcs/source/find.go

+		return
+	}
+	sp.SetTag("config.url", file.URL)
+	sp.SetTag("config", file.Content)


Isn't this too large to tag in span?

Not aware of any particular limits, infact this might be the only way for us to see those files, as we don't really have access to them if they are in a private repo.

It seems like this max_attribute_bytes config which defaults to 2048 is related?

jake-kramer · 2025-11-21T15:35:20Z

pkg/frontend/vcs/source/find.go

+	sp.SetTag("config.url", file.URL)
+	sp.SetTag("config", file.Content)
+
+	cfg, err := config.ParsePyroscopeConfig([]byte(file.Content))


nit: I still think it would be useful to load up this config when the frontend loads up and WARN any errors just as a useful sanity check. OK to instead rely on config tooling I'm working on.

Don't think this is possible, we need an end user to supply us an encrypted token and the owner/repo/root-path information before we can fetch a config.

jake-kramer · 2025-11-21T15:48:53Z

pkg/frontend/vcs/source/find.go

 // fetchRepoFile fetches the file content from the configured repository.
 func (arg FileFinder) fetchRepoFile(ctx context.Context, path, ref string) (*vcsv1.GetFileResponse, error) {
+	if arg.rootPath != "" {
+		path = filepath.Join(arg.rootPath, path)


nit: There's sort of a rough edge here where this root path is defined in the UI at repo level and then the mapping config can contain local paths that are function-level root paths. I can see as a user being confused playing with these two configs.

Maybe at some point a helper tooltip in the UI can clarify how these configs interact.

Yes I agree, in order to make the UX workable, we need to show the parsed config in context to what else is configured. I don't think though this will be a good investment of time just yet.

At least of the paths in .pyroscope.yaml are relative to that file, and the only location we are reading .pyroscope.yaml from is the "rootPath".

pkg/frontend/vcs/source/find_java.go

Also update the script for `.pyroscope.yaml` (was missed in #4606)

simonswine changed the title ~~20251111 experiment pyroscope yaml~~ WIP: Experiment with .pyroscope.yaml Nov 11, 2025

simonswine force-pushed the 20251111_experiment-pyroscope-yaml branch from 1a8e75f to fa7f045 Compare November 13, 2025 11:57

jake-kramer reviewed Nov 13, 2025

View reviewed changes

simonswine force-pushed the 20251111_experiment-pyroscope-yaml branch from d5ab32d to 85d78f1 Compare November 14, 2025 17:22

simonswine added 12 commits November 21, 2025 09:43

Add two exemplary pyroscope.yaml files

344c24f

Add some config draft code

7493226

Refactor AI slop

ee300c7

Fix some bugs

0ceb01b

Fix cleanup function

bab5ab5

Handle local with source repo correctly

711ef9c

Spring boot is a mess

78feb09

Golang stdlib impovements

4b57fe9

- Support golang.org/toolchain paths - Use mappings if there is no version identified

Support assembler file endings for go

9f22075

Instrument github errors better

d401167

Add test cases

b0f24e1

Merge both mocks

7d5ab4e

simonswine force-pushed the 20251111_experiment-pyroscope-yaml branch from e34c558 to 7d5ab4e Compare November 21, 2025 09:45

simonswine changed the title ~~WIP: Experiment with .pyroscope.yaml~~ feat: Add .pyroscope.yaml with source code mapping support Nov 21, 2025

simonswine marked this pull request as ready for review November 21, 2025 11:43

simonswine requested review from a team, bryanhuhta and marcsanmi as code owners November 21, 2025 11:43

bryanhuhta approved these changes Nov 21, 2025

View reviewed changes

pkg/frontend/vcs/config/config.go Show resolved Hide resolved

pkg/frontend/vcs/config/config.go Show resolved Hide resolved

pkg/frontend/vcs/source/find.go Show resolved Hide resolved

jake-kramer approved these changes Nov 21, 2025

View reviewed changes

simonswine added 3 commits November 24, 2025 13:49

Support version argument

1d441c2

Use multiple mappings in findJavaFile

6049370

Add a note on ExtAsm

fba4103

simonswine enabled auto-merge (squash) November 24, 2025 14:03

Add version .pyroscope.yaml

ac86cc1

simonswine merged commit 8d588a4 into main Nov 24, 2025
21 checks passed

simonswine deleted the 20251111_experiment-pyroscope-yaml branch November 24, 2025 14:17

simonswine added a commit that referenced this pull request Nov 24, 2025

Update go version to 1.24.10

e69d980

Also update the script for `.pyroscope.yaml` (was missed in #4606)

simonswine mentioned this pull request Nov 24, 2025

Update go version to 1.24.10 #4634

Open

feat: Add .pyroscope.yaml with source code mapping support #4606

feat: Add .pyroscope.yaml with source code mapping support #4606

Conversation

simonswine commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bryanhuhta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jake-kramer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: Add `.pyroscope.yaml` with source code mapping support #4606

feat: Add `.pyroscope.yaml` with source code mapping support #4606

simonswine commented Nov 11, 2025 •

edited

Loading